Course Notes for Advanced Probabilistic Machine Learning

These are lecture notes for the seminar ELEN E9801 Topics in Signal Processing: “Advanced Probabilistic Machine Learning” taught at Columbia University in Fall 2014. They are transcribed almost verbatim from the handwritten lecture notes, and so they preserve the original bulleted structure and are light on the exposition. Some lectures therefore also have a certain amount of reiterating in them, so some statements may be repeated a few times throughout the notes. The purpose of these notes is to (1) have a cleaner version for the next time it’s taught, and (2) make them public so they may be helpful to others. Since the exposition comes during class, often via student questions, these lecture notes may come across as too fragmented depending on the reader’s preference. Still, I hope they will be useful to those not in the class who want a streamlined way to learn the material at a fairly rigorous level, but not yet at the hyper-rigorous level of many textbooks, which also mix the fundamental results with the fine details. I hope these notes can be a good primer towards that end. As with the handwritten lectures, this document does not contain any references. The twelve lectures are split into two parts. The first eight deal with several stochastic processes fundamental to Bayesian nonparametrics: Poisson, gamma, Dirichlet and beta processes. The last four lectures deal with some advanced techniques for posterior inference in Bayesian models. Each lecture was between 2 and 2-1/2 hours long.